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We present a quantum information theory that allows for a con- 
sistent description of entanglement. It parallels classical (Shannon) 
information theory but is based entirely on density matrices (rather 
than probability distributions) for the description of quantum en- 
sembles. We find that quantum conditional entropies can be nega- 
tive for entangled systems, which leads to a violation of well-known 
bounds in Shannon information theory. Such a unified information- 
theoretic description of classical correlation and quantum entan- 
glement clarifies the link between them: the latter can be viewed 
as "super-correlation" which can induce classical correlation when 
considering a tripartite or larger system. Furthermore, negative en- 
tropy and the associated clarification of entanglement paves the 
way to a natural information-theoretic description of the measure- 
ment process. This model, while unitary and causal, implies the 
well-known probabilistic results of conventional quantum mechan- 
ics. It also results in a simple interpretation of the Kholevo theorem 
limiting the accessible information in a quantum measurement. 
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1 Introduction 



The recent vigorous activity in the fields of quantum information processing 
(quantum computation) and quantum communication (quantum cryptogra- 
phy, teleportation, and superdense coding) has necessitated a better under- 
standing of the relationship between classical and quantum variables (see, e.g., 
[3,10,11,17]). In classical physics, information processing and communication 
is best described by Shannon information theory [23], which succinctly asso- 
ciates information with randomness shared by two physical ensembles. Quan- 
tum information theory on the other hand is concerned with quantum bits 
(qubits) rather than bits, and the former obey quantum laws quite different 
from the classical physics of bits that we are used to [21]. Most importantly, 
qubits can exist in quantum superpositions, a notion essentially foreign to 
classical mechanics, or even classical thinking. To accommodate the relative 
phases in quantum superpositions, quantum information theory must be based 
on mathematical constructions which reflect these: the quantum mechanical 
density matrices. The central object of information theory, the entropy, has 
been introduced in quantum mechanics by von Neumann [24] 

5(p) = -IYplogp. (1) 

Its relationship to the Boltzmann-Gibbs-Shannon entropy 

H{p) = -Y^pdogpi (2) 

i 

becomes obvious when considering the von Neumann entropy of a mixture 
of orthogonal states. In this case, the density matrix p in (1) contains clas- 
sical probabilities Pi on its diagonal, and S{p) = H{p). In general, however, 
quantum mechanical density matrices have off-diagonal terms, which, for pure 
states, reflect the relative quantum phases in superpositions. 

In classical statistical physics, the concept of conditional and mutual proba- 
bilities has given rise to the deflnition of conditional and mutual entropies. 
These can be used to elegantly describe the trade-off between entropy and 
information in measurement, as well as the characteristics of a transmission 
channel. For example, for two statistical ensembles A and B, the measurement 
of (variables of) ^ by S is expressed by the equation for the entropies 

H{A)^H{A\B) + H{A:B) . (3) 

Here, H[A\B) is the entropy of A after having measured those pieces that 
become correlated in B [thereby apparently reducing H{A) to H{A\B)], while 
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H(A:B) is the information gained about A via the measurement of B. As 
is well-known, H{A\B) and H{A:B) compensate each other such that H{A) 
is unchanged, ensuring that the second law of thermodynamics is not vio- 
lated in a measurement in spite of the decrease of H{A\B) [16]. Mathemat- 
ically, H{A\B) is a conditional entropy, and is defined using the conditional 
probability pi\j and the joint probability pij describing random variables from 
ensembles A and B : 

H{A\B)^-J2Pij^ogpnj. (4) 

The information or mutual entropy (or correlation entropy) H{A:B), on the 
other hand is defined via the mutual probability pi.,j = PiPj/pij as 

H{A:B) ^ -J2Pij^ogPi..j . (5) 

Simple relations such as Pij = Pi\j pj imply equations such as (3) and all the 
other usual relations of classical information theory. Curiously, a quantum in- 
formation theory paralleling these constructions has never been attempted. 
Rather, a "hybrid" procedure is used in which quantum probabilities are in- 
serted in the classical formulae of Shannon theory, thereby losing the quantum 
phase crucial to density matrices (see, e.g., [29]). Below, in Section 2, we show 
that a consistent quantum information theory can be developed that parallels 
the construction outlined above, while based entirely on matrices [6]. This 
formalism allows for a proper information-theoretic description of quantum 
entanglement, unified with the standard description of classical correlations, 
as shown in Section 3. As a result, most of the classical concepts involving 
entropies for composite systems in Shannon theory can be extended to the 
quantum regime, and this provides a simple intuitive framework for dealing 
with quantum entropies. In the fourth section, we analyze quantum measure- 
ment in this information-theoretic language and point out how this picture 
leads to a unitary and causal view of quantum measurement devoid of wave 
function collapse [7]. In Section 5, we analyze Bell-type measurements in terms 
of information, as an application of this model. In Section 6, we conclude by 
considering a simple quantum information-theoretic derivation of the Kholevo 
theorem (which limits the amount of information that can be extracted in a 
measurement). 



2 Quantum information theory 

Let us consider the information-theoretic description of a bipartite quantum 
system AB. A straightforward quantum generalization of Eq. (4) suggests the 
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definition 



S{A\B) = -Tl AB\pABlogpA\B] (6) 

for tlie quantum conditional entropy. In order for sucli an expression to liold, 
we need to define tlie concept of a "conditional" density matrix, 



which is the analogue of the conditional probability p^j. Here, 1a is the unit 
matrix in the Hilbert space for A, (g) stands for the tensor product in the joint 
Hilbert space, and 

Pb = TyaIpab] (8) 

denotes a "marginal" or reduced density matrix, analogous to the marginal 
probability pj = J2iPij- The symmetrized product involving the infinite limit 
in the definition of the conditional density matrix (7) is a technical requirement 
due to the fact that joint and marginal density matrices do not commute in 
general. This definition for pa\b implies that the standard relation 

S{A\B) = S{AB) - S{B) (9) 

holds for the quantum entropies and that S{A\B) is invariant under any uni- 
tary transformation of the product form Ua ®Ub- More precisely, the condi- 
tional density matrix pa\b as defined by Eq. (7) is a positive Hermitian oper- 
ator in the joint Hilbert space, whose spectrum is invariant under Ua®Ub- 
However, in spite of the apparent similarity between the quantum definition 
for S{A\B) and the standard classical one for H{A\B)^ dealing with matri- 
ces (rather than scalars) opens up a quantum realm for information theory 
exceeding the classical one. The crucial point is that, while Pi\j is a probabil- 
ity distribution in i (i.e., < Pi\j < 1), its quantum analogue pa\b is not a 
density matrix: while Hermitian and positive, it can have eigenvalues larger 
than one, and, consequently, the associated conditional entropy S{A\B) can 
be negative. Only such a matrix-based formalism consistently accounts for the 
well-known non-monotonicity of quantum entropies (see, e.g., [25]). In other 
words, S{A\B) < means that it is acceptable, in quantum information the- 
ory, to have S{AB) < S{B), i.e., the entropy of the entire system AB can 
be smaller than the entropy of one of its subparts B, a situation which is of 
course forbidden in classical information theory. This happens for example in 
the case of quantum entanglement between A and B, as will be illustrated 
below [6]. 



Pa\b 



lim 



4 



The "non-classical" spectrum of the conditional density matrix pa\b is re- 
lated to the question of the separability of the mixed state Pab- First, the 
concavity of S{A\B), a property related to strong subadditivity of quantum 
entropies [25], implies that any separable state 

Pab ^^Wk p^A ® Pb^ (with ^ Wfe = 1) (10) 



is associated with a non- negative S{A\B). (Note that the converse is not true.) 
Indeed, each product component p^"* p^^ of a separable state is associated 
with the conditional density matrix 

Paib = ® 1b (11) 



so that we have 

S{A\B)>Y,w,S{p^^^)>0. (12) 



This shows that the non- negativity of conditional entropies is a necessary con- 
dition for separability. This condition can be shown to be equivalent to the 
non- violation of entropic Bell inequalities [8]. Secondly, it is easy to check 
from Eq. (6) that, if S{A\B) is negative, pa\b must admit at least one "non- 
classical" eigenvalue {i.e., an eigenvalue exceeding one), while the converse 
again does not hold. This results from the fact that Tr(pa) > if p and a are 
positive Hermitian matrices. This suggests the conjecture that a strong neces- 
sary condition for separability is that all the eigenvalues of pa\b are "classical" 
(< 1). When applied to the case of a Werner state (an impure singlet state), 
this separability condition turns out to be necessary (and sufficient for a 2 x 2 
Hilbert space) [6], since it reduces exactly to the condition derived in Ref. [19] 
by considering the positivity of the partial transpose of pab- When applied 
to a randomly generated mixture of product states, this condition is always 
fulfilled (i.e., all the eigenvalues of pa\b and Pb\a are < 1). This opens the 
possibility that it could be a stronger necessary (or perhaps even sufficient) 
condition for separability in a Hilbert space of arbitrary dimensions. Further 
work will be devoted to this question. 

Similarly to what we have done for the conditional entropy, the quantum 
analogue of the mutual entropy can be constructed by defining a "mutual" 
density matrix 



PA:B = hm \{pA «) Pb)'/>ab1" , (13) 
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the analogue of the mutual probability pi;j. As previously, this definition im- 
plies the standard relation 

S{A:B) = S{A) - S{A\B) = S{A) + S{B) - S{AB) (14) 

between the quantum entropies. This definition extends the classical notion 
of mutual informafion or correlation entropy H{A\B) to the quantum notion 
of mutual entanglement S{A:B). Note that all the above quantum definitions 
reduce to the classical ones for a diagonal pab, which suggests that Eqs. (7) 
and (13) are reasonable assumptions. (It is possible that other definitions of 
Pa\b and p^-.s can be proposed, but we believe this choice is simplest.) The 
proposed matrix-based information theory therefore includes Shannon theory 
as a special case, while it describes quantum entanglement as well. Since the 
definition of mutual entanglement S{A:B) covers classical correlations also, 
S(A:B) must be considered as a general measure of correlations and "super- 
correlations" in information theory, which applies to pure as well as mixed 
states. It is worth noticing that this does not mean that the mutual entan- 
glement characterizes the purely quantum correlation between A and B (that 
part which can be purified to singlet states); rather S{A:B) does not separate 
correlation and entanglement, it is a measure of both. In this sense, S{A:B) 
differs from various definitions of the entropy of entanglement which can be 
found in the literature [4]. Finally, we show in Ref. [1] that, besides being the 
proper quantum counterpart of correlation entropy, S{A:B) also turns out to 
be a basic quantity in the search for a quantum counterpart of Shannon's fun- 
damental coding theorem on noisy channels. Indeed, our proposed definition 
for the capacity for entanglement transmission through a quantum channel is 
written as the maximum achievable mutual entanglement S{A:B), in analogy 
with the classical definition. 

As we shall see in the next section, our quantum matrix-based formalism 
can be successfully applied to the quantum entanglement of more than two 
systems by extending the various classical entropies that are defined in the 
Shannon information-theoretic treatment of a multipartite system. This ac- 
counts for example for the creation of classical correlation through quantum 
entanglement in a tripartite (or larger) system. Also, the quantum analogue 
of all the fundamental relations between classical entropies (such as the chain 
rules for entropies and mutual entropies) holds in quantum information the- 
ory and have the same intuitive interpretation, and we make extensive use 
of it in [1,7,8]. Let us close this section by suggesting a simple diagrammatic 
way of representing quantum entropies which provides intuitive insight into 
this information-theoretic description of entanglement. In the case of a bi- 
partite system, the relations between S{A), S{B), S{AB), S{A\B), S{B\A), 
and S{A:B) are conveniently summarized by a Venn-like entropy diagram, 
as shown in Fig. la. The important difference between classical and quan- 
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Fig. 1. (a) General entropy diagram for a quantum bipartite system AB. (b) Entropy 
diagrams for three cases of a system of 2 qubits: (I) independent, (II) classically 
correlated, (III) quantum entangled. 




turn entropy diagrams is that the basic inequalities relating the entropies are 
"weaker" in the quantum case, allowing for negative conditional entropies and 
"excessive" mutual entropies [6]. For example, the upper bound for the mutual 
entropy (which is directly related to the channel capacity) is 

H{A:B) < mm[H{A), H{B)] (15) 

in classical information theory, as a consequence of the inequality H[AB) > 
max[H{A),H{B)], while it is 

S{A:B) < 2 mm[S{A), S{B)] (16) 

in quantum information theory, as a result of the Araki-Lieb inequality [25] 
S{AB) > \S(A)—S{B)\. This means that a quantum channel has a capacity for 
entanglement transmission that can reach twice the classical upper bound [1,6]; 
this is apparent for instance in the superdense coding scheme [5]. 



3 Correlation versus entanglement and multipartite systems 

We show in Fig. lb the entropy diagram corresponding to three limiting cases 
of a bipartite system of two dichotomic variables (e.g., 2 qubits): independent 
variables (case I), classically correlated variables (case II), and quantum en- 
tangled variables (case III). In all three cases, each subsystem taken separately 
is in a mixed state of entropy S{A) = S{B) = 1 bit. Cases I and II correspond 
to classical situations (which can of course be described in our matrix-based 
formalism as well, using diagonal matrices), while case III is a purely quantum 
situation which violates the bounds of classical information theory [6]. Let us 
focus on case III, since cases I and II are standard. This case corresponds to 
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an EPR pairQ, characterized by the pure state 



|V^ab) = ^(|00) + |11)), 



(17) 



and, accordingly, it is associated with a vanishing total entropy S{AB) = 0. 
Using the density matrix of the joint system pab = \4'ab){4'ab\, we see that 
subpart A {or B) has the marginal density matrix 



PA = TTB[pAB] = -m{0\ + \l){l\) 



and is therefore in a mixed state of positive entropy. This purely quantum situ- 
ation corresponds to the unusual entropy diagram (-1,2,-1) shown in Fig. lb. 
That the EPR situation cannot be described classically is immediately ap- 
parent when considering the associated density matrices. The joint and the 
marginal density matrices can be written in basis {00, 01, 10, 11} as 



Pab 



1/2 1/2 


\^ 1/2 1/2 J 



Pa = Pb 




(19) 



so that we obtain for the conditional density matrixQ 

^1 1^ 




Pa\b = PABi'^A ® Pb) 




10 1 



(20) 



Plugging (19) and (20) into definition (6) immediately yields S{A\B) = — 1, 
which results in 

S{AB) = S{A) + S{B\A) = 1-1 = (21) 



^ Although we use the term "EPR state" for the wave- function (17), this state is 
in fact one of the Bell states, which are a generalization of the EPR singlet state. 
^ Note that for EPR pairs, joint and marginal density matrices commute, simpli- 
fying definitions (7) and (13). 
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Fig. 2. Ternary entropy Venn-diagram for a general tripartite system ABC. The 
component entropies are defined in the text. 




as expected. This is a direct consequence of the fact that pa\b has one "non- 
classical" (> 1) eigenvalue, 2. It is thus misleading to describe an EPR-pair (or 
any of the Bell states) as a correlated state within Shannon information the- 
ory, since negative conditional entropies are crucial to its description^. Still, 
classical correlations [with entropy diagram (0, 1, 0)] emerge when observing 
an entangled EPR pair. Indeed, after measuring A, the outcome of the mea- 
surement of B is known with 100% certainty. The key to this discrepancy lies 
in the information-theoretic description of the measurement process [7] . Antic- 
ipating the next section, let us just mention that the observation of quantum 
entangled states such as an EPR pair gives rise to classical correlations be- 
tween the two measurement devices while keeping the entanglement between 
the two parties (particle + measurement device) unchanged, thereby creating 
the confusion between entanglement and correlation. 

More generally, the concept of negative conditional entropy turns out to be 
very useful to describe multipartite quantum systems, and it gives new in- 
sight into the creation of classical correlations from quantum entanglement. 
In the case of a tripartite system, the quantum entropies involved can be rep- 
resented by a Venn-like diagram, as shown in Fig. 2. The conditional entropies 
S{A\BC), S{B\AC), and S{C\AB) are a straightforward generalization of con- 
ditional entropies in a bipartite system, i.e., S{A\BC) = S{ABC) — S{BC), 
etc. The entropies S{A:B\C), S{A:C\B), and S{B:C\A) correspond to condi- 
tional mutual entropies. They characterize the mutual entanglement between 
two of the subsystems when the third is known. In perfect analogy with the 
classical definition, one can write, for example, 

S{A:B\C) = S{A\C) - S{A\BC) . (22) 



^ In Ref. [6], we suggest that EPR pairs are better understood in terms of a qubit- 
antiqubit pair, where the qubit (antiqubit) carries plus (minus) one bit, and where 
antiqubits are interpreted as qubits traveling backwards in time. 
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Fig. 3. (a) Ternary entropy diagram for an "EPR-triplet" or GHZ state, (b) Entropy 
diagram for subsystem AB unconditional on C. 

(a) . _ (b) . 





This is a straightforward generahzation of Eq. (14) where all the entropies 
are conditional on C. A trivial calculation gives also the expression of the 
conditional mutual entropy in terms of total entropies 

S{A:B\C) = S{AC) + S{BC) - S{C) - S{ABC) (23) 



This last expression illustrates that the conditional mutual entropies are al- 
ways non-negative as a consequence of strong subadditivity of quantum en- 
tropies (see, e.g., [25]), a property that will be useful in the following. The 
entropy in the center of the diagram is a ternary mutual entropy, defined as 

S{A:B:C) = S{A:B) - S{A:B\C) (24) 



(this generalizes Eq. (14) for a mutual entropy rather than a total entropy). 
Using Eq. (23), this can be written in a more symmetric way as 



S{A:B:C) = S{A) + S{B) + S{C) - S{AB) - S{AC) - S{BC) 

+ S{ABC). (25) 

More generally, relations between entropies in a multipartite system can be 
written, such as the "chain rules" for quantum entropies 

S{A, ■■■Ar,)^ S{A^) + S{A2\A,) + S{As\A,A2) + ■■■ (26) 



or for quantum mutual entropies 

S(Ai ■ ■ ■ A^:B) = S{Ai:B) + S{A2:B\Ai) + S(A3:B\A^A2) + ■■■ (27) 
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Let us consider as an illustration a tripartite system ABC in a Greenberger- 
Horne-Zeilinger (GHZ) [13] stateQ, 

I^ABc) = ^(|000) + |lll)). (28) 

As it is a pure (entangled) state, the total entropy is S{ABC) = 0. The 
corresponding ternary entropy diagram of ABC is shown in Fig. 3a. Note 
that the vanishing ternary mutual entropy 

S{A:B:C) = (29) 



in the center of the diagram is generic to any entangled tripartite system in a 
pure state [7]0. Indeed, the Schmidt decomposition of the pure state lipABc) 
implies that SiAB) = S{C), S{AC) = S{B), and S{BC) = S{A). This feature 
will be important in the following section as it implies that no information (in 
the sense of Shannon theory) is extracted in the measurement of a pure state. 
Fig. 3a shows clearly that, grouping say A and B (considering them as a single 
entity) results in entanglement [diagram (-1,2,-1)] between AB and C. On the 
other hand, when tracing over the degree of freedom associated with C, say, 
the resulting marginal density matrix for subsystem AB is 

PAB = TrcipABc] = ^(|00)(00| + |11)(11|) , (30) 



corresponding to a classically correlated system [diagram (0,1,0)]. As the den- 
sity matrix fully characterizes a quantum system, subsystem AB (uncondi- 
tional on C, i.e., ignoring the existence of C) is in this case physically indis- 
tinguishable from a statistical ensemble prepared with an equal number of |00) 
and 1 11) states. Thus, A and B are correlated in the sense of Shannon theory 
if C is ignored. The "tracing over" operation depicted in Fig. 3b illustrates 
this creation of classical correlation from quantum entanglement. In short, the 
EPR-triplet entails quantum entanglement between any part, e.g. C, and the 
rest of the system AB. The subsystem AB unconditional on C has a positive 
entropy S{AB) of 1 bit, and is indistinguishable from a classical correlated 
mixture. On the other hand, the entropy of C conditional on AB, S{C\AB), 
is negative and equal to —1 bit, thereby counterbalancing S{AB) to yield a 
vanishing combined entropy 

S{ABC) = S{AB) + S{C\AB) = , (31) 

^ The GHZ state can also be viewed as an "EPR-triplet", a generalization of an 
EPR-pair to three parties. 

^ For a multipartite system, the mutual entropy between the n parts is equal to 
1 + (-1)". 
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as expected in view of the entanglement between AB and C. The above can 
be extended in a straightforward manner to composite systems, and this will 
be central to the measurement process. 



4 Quantum measurement 

According to von Neumann [24] , a consistent description of the measurement 
process must involve the interaction between the observed quantum system 
and a quantum measurement device. Such a view is in contrast with the Copen- 
hagen interpretation of quantum mechanics (see, e.g., [26]) stating that the 
measurement is the non-causal process of projecting the wave-function, which 
results from the interaction with a classical apparatus. A classical appara- 
tus is defined as one where the "pointer" variables take on definite values, 
and which therefore cannot refiect quantum superpositions. For 70 years, the 
Copenhagen interpretation has never failed in predicting a single experimen- 
tal fact, which certainly has helped in cementing its reputation [26]. On the 
other hand, if the foundations of quantum mechanics are believed to be solid, 
it cannot be denied that measurement is not an abstract non-causal operation 
acting on wave functions, but rather a genuine interaction between two physi- 
cal quantum systems: the observed system Q and the measurement device, or 
the ancilla A. This is the essence of the von Neumann theory of measurement. 

Assume then that a quantum system is initially in state 



expressed in the basis {|ai)} of eigenvectors of an arbitrary observable (the one 
that we are measuring). Then, the von Neumann measurement is described 
by the unitary transformation that evolves the initial state of the joint system 
\Q,A) = \Q,Q) into the state 



with denoting the eigenstates of the ancilla (|0) is the reference initial 

state of the ancilla). Such a transformation was interpreted by von Neumann 
as inducing correlations between the system Q and the ancilla A. Indeed, if \Q) 
is initially in one of the eigenstates |aj) (i.e., if it is not in a superposition), the 
"pointer" in A that previously pointed to zero now points to the eigenvector |i) 
which labels outcome i, suggesting that a measurement has been performed. 

Now, a basic problem occurs if the initial state of Q is a superposition, as 




(32) 




(33) 
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in Eq. (32), that is, if Q is not in an eigenstate of the considered observable. 
Then, according to Eq. (33), the apparatus apparently points to a superpo- 
sition of i's, a fact which obviously contradicts our everyday-life experience. 
In classical physics, a variable has, at any time, a definite value that can be 
recorded. Experiments show that a quantum measurement is probabilistic in 
nature, that is one of the possible outcomes (drawn from a probability dis- 
tribution) becomes factual. In other words, a quantum superposition evolves 
into a mixed state. This apparent necessity led von Neumann to introduce 
an ad hoc, non-unitary, second stage of the measurement, called observation. 
In this stage, the measurement is "observed" , and a collapse occurs in order 
to yield a classical result from a quantum superposition. The central point in 
the quantum information-theoretic interpretation of the measurement prob- 
lem presented below (see also Ref. [7]) is that, in general, the state described 
by Eq. (33) is entangled., not just correlated. As emphasized earlier, entan- 
gled states have an information-theoretic description distinct from correlated 
states, which provides them with very peculiar properties. For example, it has 
been shown that an arbitrary quantum state cannot be cloned [27] precisely 
because of the entanglement between the system Q and the ancilla A. If the 
system is in a state belonging to a set of orthogonal states, on the other hand, 
a faithful copy of the quantum state can be obtained applying a von Neumann 
measurement. As a consequence it appears that an arbitrary state (one which 
is not one of the eigenstates of the observable considered) can not be measured 
without creating entanglement. 



Let us show that unitary evolution [such as the one giving rise to Eq. (33)] can 
be reconciled with the creation of randomness in the measurement process if 
it is recognized that the creation of entanglement (rather than correlation) is 
generic to a quantum measurement, and if this entanglement is properly de- 
scribed in quantum information theory using the concept of negative entropy 
(see also Ref. [7]). This reconciliation is brought about by a redescription of 
the second stage of measurement, the observation, without involving an irre- 
versible loss of information to a macroscopic environment. In that respect, our 
model is distinct from the environment-induced decoherence model, one of the 
prevalent contemporary views of quantum measurement (see e.g. [30]). In or- 
der to observe the measurement, a system involving generally a large number 
of degrees of freedom has to interact with Q. In Eq. (33), Q has interacted 
with a single degree of freedom of the ancilla A (first stage of the measure- 
ment), which led to an entangled state. As emphasized before, the creation of 
an entangled state does not mean that a measurement has been performed, 
since our (classical) perception of a measurement is intrinsically related to 
the existence of (classical) correlations. In order for classical correlations to 
emerge, a third degree of freedom (another ancilla A') has to be involved. 
Now, iterating the von Neumann measurement. A' interacts with AQ so that 



13 



the resulting state of the combined system is 

\QAA) =Y,a,\a,,i,i) , (34) 

i 

where the eigenstates of A' are also denoted by \i) for simplicity. The state so 
created is pure [S{QAA') = 0], akin to an "EPR-triplet" since the system has 
undergone only unitary transformations from a pure initial state IQ, 0, 0). The 
point is that, considering the state of the entire ancilla A A' unconditionally 
on system Q yields a mixed state 

Paa' = Txq\pqaa'] = X] (35) 

i 

describing maximal correlation between A and A\ that is 

S{A:A') = S{A) - S{A') - S{AA') . (36) 

The second stage consists in observing this classical correlation (that extends, 
in practice, to the 10^"^ particles which constitute the macroscopic measure- 
ment device). Note that a macroscopic measurement device is not required 
here, since only two ancillary degrees of freedom A and A' are enough to 
generate correlation in the tripartite entangled system QAA' . The entropy di- 
agram characterizing QAA' is of the same kind as the one depicted in Fig. 3, 
but filled in with a constant different from 1 in general. Paradoxically, it is the 
physical state of the ancilla which contains the outcome of the measurement, 
whereas the quantum state Q itself must be ignored to observe correlations. 
This crucial point is easily overlooked, since intuition dictates that perform- 
ing a measurement means somehow "observing the state of Q". Rather, a 
measurement is constructed such as to infer the state of Q from that of the 
ancilla — but ignoring Q itself. The correlations (in A A') which emerge from 
the fact that a part [Q] of an entangled state {QAA') is ignored give rise to 
the classical idea of a measurement. This view of the measurement process 
insists only on the "self-consistency" of the measurement device, while aban- 
doning the 100% correlation between the latter and the quantum system Q, 
a cornerstone of decoherence models. More precisely, no information (in the 
sense of Shannon theory) about Q is obtained from the ancilla. Indeed, using 
Eqs. (23) and (36), we have 

S{Q:A:A') = S{A:A') - S{A:A'\Q) = (37) 

meaning that the mutual entropy between A and A' (the observed correlation) 
is not shared with Q. This is a consequence of the fact that Q is initially in a 
pure state. We will see in the next section that information (Shannon mutual 
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entropy) can only be acquired in the situation where a mixed state is measured. 
After measurement, the quantum entropy of the ancilla (unconditional on Q) 



S{AA') = H\pi] with Pi = \ai 



(38) 



is interpreted as the "physical" entropy of Q. This happens to be the classical 
entropy associated with the probability distribution of the random outcomes, 
Pi — lo^jP) that is the probabilities predicted by quantum mechanics. Thus 
the unconditional entropy of the ancilla is equal to the entropy of Q predicted 
in "orthodox" quantum mechanics (which involves the projection of the wave 
function). Still, the entropy of Q conditional on AA' is negative, and exactly 
compensates S{AA') to allow for a vanishing entropy for the joint system. 



This then emphasizes how measurement can be probabilistic in nature, while 
at the same time being described by a unitary process (which does not permit 
the evolution of pure into mixed states). 

The appearance of a wave-function collapse, crucial in the physics of sequential 
measurements, can also be interpreted in this information-theoretic picture. If 
a second ancilla B (in general, also a large number of degrees of freedom) inter- 
acts with Q in order to measure the same observable (after a first measurement 
involving ancilla A), the result is an "EPR-nplet" (consisting of all the degrees 
of freedom of A, B, and the measured quantum state Q). To simplify, let us 
consider two ancillary variables A and B (and neglect their amplification). 
Then, the final quantum state after the sequential measurement is 



illustrating clearly that the state of A and B (unconditional on Q) are clas- 
sically maximally correlated just as described earlier. This is the basic consis- 
tency requirement for two consecutive measurements of the same variable: we 
must have S{B\A) = 0. The standard assertion of orthodox quantum mechan- 
ics is that, after the first measurement, the wave function of Q is projected 
on \ai), the observed eigenstate, so that any following measurement yields the 
same value i without any remaining uncertainty since the state of Q is now 
\ai). As we just showed, such a classical correlation between the outcome of two 
measurements actually involves no collapse; rather, the vanishing remaining 
uncertainty of the second measurement [refiected by the vanishing conditional 
entropy S{B\A) = 0] is due to the fact that one considers only part of an 
entangled system. 



S(AA') + S(Q\AA') = S(QAA') = 



(39) 




(40) 
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More interestingly, in the case where the ancilla B measures another observ- 
able, Eq. (40) becomes then 

\QAB)=J2a,U,,\b„t,j) (41) 

where (l&j)} are the eigenvectors of the second observable, Uij = {bj\ai), and 
{\j)} denote the eigenstates of the second ancilla B. The resulting entropy dia- 
gram for AB (obtained by tracing over Q) gives rise to an entropic uncertainty 
relation [7] 

S{A) + S{B) >mmH 

i 

where the right-hand side stands for the Shannon entropy of the probability 
distribution resulting from the expansion of the eigenvector |aj) of the first ob- 
servable into the eigenbasis of the second observable, minimized over i. This 
reflects the fact that the sequential measurement of two non-commuting ob- 
servables (such that Uij is not the identity or a permutation matrix) must 
generate a non-zero entropy. Eq. (42) is compatible with the uncertainty rela- 
tions found in the literature (see, e.g., [9]), but can be shown to be stronger in 
intermediate situations between compatible and complementary (maximally 
incompatible) observables [7]. 



5 Bell-type measurements 

In order to illustrate the information-theoretic analysis of measurement de- 
scribed above, let us consider the measurement of an EPR pair. This should 
also clarify how quantum entanglement can have the appearance of classical 
correlation in such an experiment. Let us prepare a bipartite system Q1Q2 in 
the EPR-entangled state 

\Q1Q2) = ^(1 TT) + I ii)) (43) 

and separate the two members at remote locations in space. At each location, 

the system {Qi or Q2) is measured by interacting with an ancilla {Ai or 
A2), following the same procedure as before. In brief, each system {Qi or Q2) 
becomes entangled with its corresponding ancilla, resulting in the entangled 
state 

\QiQ2A,A2) = 4^(1 TT 11) + I ii 00)) (44) 



(42) 
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Fig. 4. Ternary entropy diagram for the measurement of an EPR pair, (a) Ai and 
A2 both measure the spin z-component cr^- (b) Ai measures az while A2 measures 




for the entire system. Note that an ancilla in state |1) means that a spin-up 
has been measured, and conversely. (Obviously, this corresponds to the mea- 
surement of the spin-projection along the z-axis; the measurement of different 
spin-components of Qi and Q2 can be considered along the same lines.) As 
previously, we describe the ancilla with just one internal variable, even though 
in practice it must be thought of as consisting of a large number of correlated 
ones. The important point here is that, despite the fact that Qi and Q2 were 
initially in an entangled state [characterized by the (—1,2,-1) entropy dia- 
gram], the state of the two ancillae unconditional on Qi and Q2 is a mixed 
(classically correlated) state 

= ^(|00)(00| + |11)(11|) (45) 

Thus, the ancillae are correlated: the corresponding entropy diagram (0,1,0) 
clearly shows that, after observing Ai, for instance, the state of A2 can be in- 
ferred without any uncertainty, i.e., S{A2\Ai) = 0. However, this must not be 
attributed to the existence of classical correlation between Qi and Q2', rather 
it is the act of measuring which gives rise to this appearance. The entropy 
relations between Qi, Q2, Ai and A2 can be summarized by an entropy di- 
agram (Fig. 4a). This emphasizes that it is the same mechanism which is at 
the origin of the coincidence between the observed spin-projection for both 
particles in an EPR experiment and at the core of the consistency between se- 
quential measurements on a single quantum system.^ In the former case, the 
mechanism is well known and accepted, while in the latter case, it is more dif- 
ficult to discern and an ad hoc collapse is therefore often wrongly invoked. For 
completeness, the entropy diagram describing the situation where the ancillae 
Ai and A2 measure orthogonal spin projections {e.g., ax and a^) is shown in 
Fig. 4b. When tracing over Qi and Q2, it is obvious that the ancillae Ai and A2 
are statistically independent, which accounts for the fact that two apparently 
independent random variables {ax and az) are measured. In reality, the entire 
system is entangled in a particular way: Ai and A2 are entangled separately 

^ We thank Zac Walton for pointing this out to us. 
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with QiQ^- Note finally that the violation of Bell inequalities occurring in the 
measurement of EPR pairs can also be analyzed from an information-theoretic 
point of view, as shown in Ref. [8]. 



6 Measurement of mixed states and accessible information 

An important issue of quantum information theory is the maximum amount 
of information that one can extract about a quantum system by performing 
a measurement. Let us consider a simple derivation of this quantity based on 
conditional and mutual quantum entropies and on simple relations between 
them. This derivation, akin to the proof of Schumacher et al. [22], relies on 
our information-theoretic description of unitary measurement and does not 
involve any "environmental" degrees of freedom (it does not involve decoher- 
ence induced by an environment [30]). As emphasized before, the entropy that 
appears in the ancilla A is "extracted" from the measured quantum system Q, 
whose conditional quantum entropy therefore becomes negative. This means 
that the quantum system and the ancilla are entangled as a result of the mea- 
surement, and that the measurement simply becomes the "act" of ignoring - 
or tracing over - the quantum system Q which is aimed to be measured. This 
is in contrast with the prevalent view of measurement, where the quantum 
system and the ancilla become classically correlated because one is compelled 
to ignore the numerous degrees of freedom of an uncontrollable environment 
(in other words, decoherence leads to the selection of a "preferred basis" for 
the "pointer variable" [30]). As stressed in Section 4, the appearance of a 
collapse of the wave function can be fully understood when considering sub- 
sequent measurements without any environment; the statistics of the state of 
the ancillae that interact sequentially with the quantum system Q is indistin- 
guishable from the statistics resulting from the collapse postulate. 

A striking consequence of this information-theoretic interpretation is that, in 
any measurement of a pure state, no information at all (in the sense of Shan- 
non theory) can possibly be extracted from the system. In other words, no 
information is gained about the identity of the pure state. (This means that 
the "pointer variable" is not classically correlated with the quantum system.) 
Recognizing that a pure state has a vanishing von Neumann entropy, this turns 
out to be an obvious result: there is no uncertainty about it, so nothing can be 
learned from it. This can also be understood as a consequence of the quantum 
non-cloning theorem [27]: one cannot "clone" {i.e., correlate in the Shannon 
sense) an arbitrary state with an ancilla, as only entanglement results from 
the measurement. It is also straightforward to see, by looking at quantum en- 
tropies, that the correlations that appear in a measurement do not concern Q 
vs. A but rather concern all the pieces of the (generally macroscopic) ancilla: 



18 



the ancilla is "self-consistent" Q. Clearly, as far as information extraction is 
concerned, a more interesting case to consider is the measurement of a quan- 
tum system Q initially prepared in a mixed state; only then can information 
be extracted about the preparation of the state. 

A measurement performed on a quantum system which can be prepared in 
different states yields an amount of information about the preparation which 
is limited by the Kholevo bound [15]. More precisely, if a system is prepared 
in a state described by one of the density operators pi {i = 1, ■ ■ -n), with 
probability pi, then the information / that can be gathered about the identity 
of the state is always lower than the Kholevo bound 

I<S{J2P^P^)-J2P^Sip^). (46) 

i i 

This result holds for any measurement one can perform on the system, includ- 
ing positive-operator- valued measures (POVM's). Since the original conjec- 
ture by Kholevo, a lot of effort has been devoted to obtaining a more rigorous 
proof of the theorem, or to derivations of stronger upper or lower bounds on I 
[18,20,28,14,12,22]. Our aim here is to give a simple proof of this upper bound 
on accessible information which is based on quantum entropies, as opposed to 
deriving Shannon entropies from the quantum probabilities associated with 
measurements, as is usually done. The derivation relies only on the unitarity 
of the measurement seen as a physical process, along with the strong subad- 
ditivity property of quantum entropies (c/. Sect. 3). This makes the physical 
content of the Kholevo theorem more transparent: in short, it states that the 
classical mutual entropy (i.e. the acquired information J) is bounded from 
above by a quantum mutual entropy. 

Let us assume that we have a "preparer", described by a (discrete) internal 
variable X, which is distributed according to the probability distribution pi 
[i = 1,---N). The internal state of the preparer, considered as a physical 
quantum system, is given by the density matrix[^ 

Px = '^Pi\xi){xi\ (47) 

i 

with the \xi) being an orthonormal set of states. The state of the quantum 
variable X can be copied to another system simply by making conditional 

^ If Ai and A2 represent two halves (arbitrarily chosen) of the ancilla, the ternary 
mutual entropy S{Ai:A2.Q) vanishes if the quantum system Q is initially in a pure 
state and if the measurement process is unitary. But, the ancilla is "self-consistent" 
that is 5(^21^1) = S{Ai\A2) = 

^ Of course, px can be seen as resulting from the partial trace of a pure state in 
an extended Hilbert space (it can be "purified" via a Schmidt decomposition). 
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dynamics (the simplest example being a controlled-NOT quantum gate) and 
in that sense, it behaves like a classical variable (it can be "cloned"). Let 
us therefore denote by X the collective set of correlated internal variables 
describing the preparer state. Assume now that the preparer has at his disposal 
a set of N mixed states pi, and that he chooses one of them for Q according to 
his internal state X. The joint state of the preparer and the quantum system 
Q is then given by 

PxQ = ^Pi\xi){xi\® pi (48) 



and a partial trace over X simply gives the state of Q: 

Pq = TrxPxQ = J^PiPi = P ■ (49) 



The quantum entropy oi X, Q and the joint system XQ is given by 

S{X) = H[p,], 
S{Q) = Sip), 

S{XQ) = H\p,] + Y,p,S{pi), (50) 

i 

where the last expression results from the fact that pxQ is block-diagonal (it 
is the quantum analogue of the "grouping theorem" in Shannon theory [2]). 

Now, the quantum system Q is "measured" by interacting unitarily with an 
ancilla A, according to 

PX'Q'A' = (Ix ® Uqa){pxq ® |0)(0|)(lx ® Uqa^ (51) 

where |0)(0| denotes an initial reference state of the ancilla, and X', Q', and 
A' correspond to the respective systems after the unitary evolution Uqa- For 
the moment, let us assume that Uqa is arbitrary. The interesting question will 
be to determine the mutual quantum entropy S(X':A') between the physical 
state of the ancilla A after measurement and the physical state of the pre- 
parer X (which remains unchanged in the measurement). We will show that, 
given certain assumptions for Uqa, S{X':A') represents simply the Shannon 
mutual entropy between the preparer and the ancilla, or, in other words, the 
information I extracted by the observer about the preparer state. 

The relations between the entropies of X and Q before measurement can be 
summarized by the quantum entropy diagram in Fig. 5. It is easy to calculate 
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Fig. 5. Entropy Venn diagram for the correlated system XQ before measurement. 

X ^ ^ ^ ^ Q 




the quantum mutual entropy (or mutual entanglement) between X and Q 
before measurement, 



S{X:Q) = S{X) + S{Q) - S{XQ) = S{p) - J2piS{p, 



(52) 



showing that S{X:Q) is simply the Kholevo bound [see Eq. (46)]. Invoking the 
upper and lower bounds for the entropy of a convex combination of density 
matrices (see e.g. [25]), i.e., 



i \ i / i 



(53) 



implies 



< SiX:Q) < H{pi\ . 



(54) 



This shows that the entropy diagram for XQ (represented in Fig. 5) has 
only positive entries and therefore looks like a classical diagram for correlated 
variables!^. 

Before measurement, the ancilla A is in a pure state |0) and the joint state of 
the system XQA is a product state pxq ® |0)(0|, so that we have S{X:Q) = 
S{X:QA). As the measurement involves unitary evolution of QA and leaves X 
unchanged, it is straightforward to check that this mutual entropy is conserved: 



S{X':Q'A') = S{X:QA) = SiX:Q) . 



(55) 



Next, we may split this entropy according to the quantum analogue of the 
chain rules for mutual entropies [Eq. (27)] to obtain 



S{X':Q'A') = S{X':A') + S{X':Q'\A' 



(56) 



^ As explained earlier, this property is related to the fact that pxq is a separable 
state and therefore is associated with positive conditional entropies. 
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Fig. 6. Diagrammatic representation of the Kholevo theorem. The area enclosed 
by the double solid lines represents the mutual entropy that is conserved in the 
measurement S{X':Q'A') = S{X:Q). 




where the second term on the right-hand side is a quantum conditional mutual 
entropy {i.e., the mutual entropy between X' and Q', conditionally on A). 
Combining Eqs. (55) and (56) gives the basic relation 

S^X'-.A) = S{X:Q) - S{X':Q'\A') . (57) 

This equation is represented as arithmetic on Venn diagrams in Fig. 6. 

Thus, the quantum mutual entropy between the state of the preparer X' (we 
can ignore the prime since X is unchanged in the measurement) and the state 
of the ancilla after measurement A' is given by S{X:Q), the Kholevo bound, re- 
duced by an amount which represents the mutual entropy still existing between 
the preparer's internal variable X and the quantum state after measurement 
Q', conditional on the observed state of the ancilla A'. Since S{X':Q'\A') is 
in general difficult to calculate, we can make use of strong subadditivityP^ in 
order to obtain an inequality. In particular, we have S{X':Q'\A') > 0, which 
yields the simple upper bound: 

S{X':A') < S{X:Q) = S{p) - Y.P^S{p,) . (58) 

i 

It remains to show that, for a particular Uqa which describes a measurement, 
the quantum mutual entropy S{Q':A') reduces to a Shannon mutual entropy 
(the mutual information / between the state of the preparer and the outcome 
of the measurement). 

Let us consider only the case of a von Neumann measurement!^ using the 

Expressed in our quantum information-theoretic language, strong subadditivity 
implies that the conditional mutual entropy S{X:Y\Z) between any three quantum 
variables X, Y, and Z is non-negative. This expresses the intuitive idea that the 
mutual entanglement between X and YZ is larger or equal to the mutual entangle- 
ment between X and Z only (just as mutual informations in Shannon theory), so 
that a mutual entanglement can never decrease when extending a system. 

It can be shown that the same reasoning applies also to a positive-operator- valued 
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explicit form 

UQA = ^Pa®Va (59) 
a 

where the index a refers to the outcome of the measurement and the P^'s de- 
note the projectors in the Q space associated with the measurement {J2a Pa = 
1). The unitary operators V„ act in the A space, and move the ancilla from 
the initial state |0) to a state \a) = Va\0) that points to the outcome of the 
measurement. Let us assume that the |a) are orthogonal so that the outcomes 
are perfectly distinguishable. The joint density matrix after unitary evolution 
is thus given by 

PX'Q'A' = Pi\^i){^i\ ® PaPiPa' ® • (60) 

i,a,a' 

As before, we now have to trace over the quantum system Q' in order to induce 
correlations between X' and A'. The corresponding density matrix is 

px'A' = X!PjTr(Pc,Pi)kj)(a;i| O \a){a\ . (61) 

i,a 

As it is a diagonal matrix, the relations between the entropies of X' and A' 
can be described within Shannon theory (the quantum definitions of condi- 
tional and mutual entropies reduce to the classical ones in this case.) A simple 
calculation shows that one has indeed 

Six' -.A') = H [Tr(P,p)] - Y^piH [Tr(P,p,)] 

i 

= H{A) - H{A\X) = H{A:X) , (62) 

where Tr(P„pj) is the conditional probability Pa\i of measuring outcome a 
on states pj, so that it is justified to identify S{X':A') with the information 
/ in this case. Note that the information gained in the measurement is not 
described as a difference between initial and final uncertainty of the observer 
(involving a calculation of probabilities as it is usually done), but rather as 
a quantum mutual entropy. As a result of Eq. (62), we see that Eq. (58) 
provides an upper bound on the accessible information, and this completes our 
derivation of the Kholevo theorem. As shown elsewhere, the same reasoning 
can be extended to the case of sequential measurements of a quantum system, 
using chain rules for quantum entropies, providing a generalization of the 
Kholevo theorem. 

measure (POVM) in general. 



23 



As a final remark, let us mention that inequality (58) can be shown to be 
a special case of a more general relation. For an arbitrary density matrix 
PxY describing a bipartite quantum system whose components interact with 
ancillae A and B that define bases |a;) and \y) respectively, we have clearly 
S{A':B') = H{X:Y), where H{X-Y) is the Shannon mutual entropy of the 
joint probability = {^,y\pxY\x,y). Using 



and the non-negativity of conditional mutual entropies yields the general in- 
equality 



between classical and quantum mutual entropies. 



7 Conclusions 

We have shown that quantum entanglement can be consistently described us- 
ing the notion of negative conditional entropy, an essential feature of a quan- 
tum information theory built entirely on density matrices. Negative quantum 
entropy can be traced back to "conditional" density matrices which admit 
eigenvalues larger than unity. A straightforward definition of quantum mutual 
entropy, or mutual entanglement, can also be obtained using a "mutual" den- 
sity matrix. This quantum matrix-based formalism gives rise to the violation 
of well-known bounds in classical information theory. It treats quantum en- 
tanglement and classical correlation on the same footing, while clarifying in 
which sense entanglement can induce correlation. This last point allows for a 
consistent information-theoretic description of unitary quantum measurement, 
devoid of any assumption of wave-function collapse, which, at the same time, 
accounts for the creation of entropy (random numbers) in the measurement 
outcome. This sheds new light for example on information-theoretic aspects of 
Bell-type experiments or on the issue of how much information can be accessed 
in a quantum measurement. Also, as quantum entanglement is a central fea- 
ture of quantum computation, we believe that the present formalism will shed 
new light on decoherence (entanglement with an environment) in noisy quan- 
tum channels, as well as the error-correcting codes being devised to counteract 
it. From a more fundamental point of view, the fact that quantum conditional 
entropies can be negative reveals that quantum statistical mechanics is qual- 
itatively very different from classical statistical mechanics, even though most 
of the formulae are similar. 



s{x-y)^s{x'A!-y'B') 

^S{A':B') + S{A':Y'\B') + S{X':Y'B'\A') 



(63) 



H{X:Y) < S{X:Y) 



(64) 
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